Summary

• migrate model-engine from python:3.13-slim to public Chainguard python:latest-dev / python:latest
• update the Python dependency set for Python 3.14 compatibility and current security fixes
• preserve runtime boot by carrying the required git executable and shared libraries into the minimal runtime image

What Changed

• model-engine/Dockerfile
  • switched builder to cgr.dev/chainguard/python:latest-dev
  • switched runtime to cgr.dev/chainguard/python:latest
  • replaced the old Debian/apt-get flow with apk in the builder stage
  • removed the old Debian runtime package install path and other runtime baggage from the previous image shape
  • build the Python environment in a venv and copy only the needed runtime artifacts forward
  • copy service_configs into the image so the gateway startup path can resolve service_config_circleci.yaml
  • copy git, git-core, and the required runtime libraries (libpcre2-8, libz) from the builder stage into the final image so GitPython-backed startup imports still work
• model-engine/requirements.in
  • bump ddtrace to >=4.7.1,<5.0
  • bump numpy to >=2.4.4,<2.5
  • bump google-cloud-artifact-registry to ~=1.21.0
  • bump psycopg2-binary to 2.9.11
  • add pytz>=2024.1
  • bump pydantic to 2.12.5
• model-engine/requirements.txt
  • refresh the transitive set needed for Python 3.14 and the updated base image, including:
    • ddtrace 4.7.1
    • envier 0.6.1
    • numpy 2.4.4
    • grpcio 1.75.1
    • grpcio-status 1.75.1
    • protobuf 6.33.5
    • google-cloud-artifact-registry 1.21.0
    • psycopg2-binary 2.9.11
    • pytz 2025.2
    • pydantic 2.12.5
    • pydantic-core 2.41.5

Why

The previous model-engine image was based on python:3.13-slim and carried a large Debian OS vulnerability surface. Moving to the public Chainguard Python images materially reduces OS exposure, but because public Chainguard currently tracks Python 3.14, the repo also needed a coordinated dependency refresh to restore build and runtime compatibility.

The runtime boot path also imports GitPython-backed ECR helpers during gateway startup. A minimal runtime image therefore still needs the git executable and its required shared libraries present, even though the rest of the image is aggressively minimized.

Validation

Build-time

• docker build --platform linux/amd64 --progress=plain -f model-engine/Dockerfile -t llm-engine-chainguard-min:local .
  • passed

Runtime

• direct import smoke:
  • import model_engine_server
  • passed
• gateway boot with temporary credentials exported from the local production-developer AWS profile and explicit local test env:
  • GIT_TAG=test
  • CIRCLECI=true
  • AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN
• health check:
  • GET /healthz returned 200

Security scan

• trivy image --scanners vuln --severity CRITICAL,HIGH --format json -o /tmp/llm-engine-chainguard-min-final-trivy.json llm-engine-chainguard-min:local
• final result:
  • 0 CRITICAL
  • 0 HIGH

Notes

• The runtime still emits warning-level noise during startup, including:
  • no PyTorch / TensorFlow / Flax present in this image
  • a ddtrace warning about Pydantic v1 functionality on Python 3.14
  • a few Python syntax warnings in existing repo code
• Those warnings did not block gateway startup or the /healthz check.

Greptile Summary

This PR migrates the model-engine runtime from python:3.13-slim to the Chainguard python:latest-dev/python:latest image pair, achieving a 0 CRITICAL / 0 HIGH Trivy result, and coordinates a full Python 3.14-compatible dependency refresh (ddtrace 4.7.1, pydantic 2.12.5, protobuf 6.33.5, grpcio 1.75.1, numpy 2.4.4). It also refactors the remote Docker-build pipeline with cleaner archive filtering, a proper path-normalization helper for build args, and new unit-test coverage for previously untested helpers.

The prior round of review flagged several regressions that are now resolved — kubectl is built from source and copied into the runtime, libcurl/libreadline/libtinfo are collected alongside libz/libpcre2, TARGETARCH is threaded through both Go compilation steps, and the lstrip("./") dotfile-mangling bug is fixed with removeprefix. The BUILD_CONTEXT_TEMP_ROOT being placed inside model-engine/ (so accumulated temp dirs are included in every subsequent build archive) remains unaddressed from the prior round.

Important Files Changed

sequenceDiagram
    participant EB as EndpointBuilder Pod (Chainguard runtime)
    participant S3 as S3
    participant K8s as Kubernetes API
    participant Kaniko as Kaniko Pod

    EB->>EB: _create_build_context_dir() under WORKSPACE/model-engine/.build-context/
    EB->>EB: _normalize_build_args() rewrite abs paths to relative
    EB->>EB: zip_context() _filter_archive_member() excludes nested roots and ignore patterns
    EB->>S3: upload tar.gz build context
    EB->>K8s: kubectl patch secret codeartifact-pip-conf
    EB->>K8s: kubectl apply -f kaniko-job.yaml
    K8s->>Kaniko: start Kaniko pod
    Kaniko->>S3: download tar.gz context
    Kaniko->>Kaniko: docker build and push to ECR
    EB->>K8s: watch pod status (kubernetes Python client)
    K8s-->>EB: pod Succeeded/Failed
    EB->>K8s: kubectl logs (read final build output)

_{Reviews (22): Last reviewed commit: "fix: skip rewriting build context root a..." | Re-trigger Greptile}